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Techniques for Test Construction 


evaluation of the effectiveness of educational programs. 

The construction of tests, the development of record forms, 
and the determination of what records to keep have most fre- 
quently been the responsibility of school administrators and, in 
many cases, have been greatly influenced by individuals out- 
side the local school system. 

Where teachers share the responsibility of determining pur- 
poses, where they play an important part in determining what 
records to keep, and where they carry the major burden of the 
evaluation work, they frequently feel the need for technical 
assistance. This assistance seems to be effective to the degree 
that the objectives are clarified, that purposes are sharpened, 
and that, as a result, teachers are able to see in the everyday 
behavior of their students significant aspects of personality. 

In the early years of this venture in evaluation which is a 
part of the Eight-Year Study of the Progressive Education 
Association, much reliance was placed upon “anecdotal records.” 
These were supposed to be informal records of pupil behavior. 
The teachers were to write brief reports on things that had 
actually happened in the classroom and which they considered 
important. Teachers often complained that they did not have 
time to do this, and they raised many other objections concerning 
the reliability and validity of such records. Even more frequent 
than these complaints, however, came the questions: What shall 
we write about? What is important? To help teachers answer 
these questions for themselves, great emphasis was placed upon 
discussions of objectives; but even this did not meet the situa- 
tion because many teachers showed great verbal fluency in 
stating their purposes and little discrimination in the actual 
observation of their students. To improve in observation 
seemed to be highly correlated with continued practice in stating 
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objectives in terms of student behavior. Where the objective 
had become so concrete that it could be illustrated many times 
by activities which boys and girls actually carried out in the 
classroom, teachers had little difficulty in recognizing behavior 
that was worth writing about. This sharpening of the powers 
of observation was also increased by knowledge of the tech- 
niques of evaluating student achievement. 

In this issue of the BULLETIN, the writer has described three 
techniques for constructing evaluation instruments relating to 
different aspects of thinking. Teachers who understand these 
methods for making tests will certainly be in a better position 
to approve or disapprove their inclusion in the evaluation pro- 
gram. They will be better able to appraise many tests which 
purport to evaluate such intangibles as thinking. Even more 
important than this, however, is the hope that teachers who 
understand what behavior is to be appraised, and how that 
evaluation is to be done, will be much more alert and much 
more discriminating in their observations of students, and hence, 
much more effective in guiding their students. 


Measuring the Ability to Apply 
Scientific Principles 


Peer: of a student’s ability to apply principles pre- 


supposes some definition of the word. As herein used, 
principles are thought of as general propositions, or as 
generalizations of experience which man has made in an attempt 
to make order of the universe for purposes of prediction, expla- 
nation, or control of events. Principles vary with respect to 
their usefulness in satisfying these purposes. Many principles 
in the physical sciences are more exact and can be better used 
for predicting results than in the social sciences. In all fields, 
however, there are generalizations of some value which teachers 
hope their students will learn and apply. Since the ability to 
apply principles is common, ways of evaluating are important. 
How do students most frequently use principles? John 
Dewey says that 
intelligence is concerned with foreseeing the future so that action may 
have order and direction. It is also concerned with principles and 








Ly 


d 





APRIL 13, 1938 87 


criteria of judgment. . . . Principles are methods of inquiry and fore- 


cast which require verification by the event. . . . Principles exist as 
hypotheses with which to experiment." 


Most people will agree with Mr. Dewey that a common use of 
principles is in the prediction of events—foretelling what prob- 
ably will happen in particular circumstances. Principles are also 
commonly used to explain an event which has already hap- 
pened.” Most schools emphasize this aspect of the application 
of principles. Students are asked frequently to explain why a 
particular event happened and are further requested to discuss 
their answers in detail. It is safe to generalize that students 
make most frequent use of their knowledge in explaining why 
something happened, or in predicting what probably will hap- 
pen and then justifying that prediction or explanation with 
reasonable arguments. 

What are the pitfalls, the common errors which students 
most frequently make when the ability to apply principles is 
imperfectly developed? The following description is made 
from the judgments of many teachers and from a study of many 
essay papers written by students. First, we find students who 
do not know the principles which are involved. They were in 
the same classroom with the other students who know them, 
but they either did not learn them or learned them and forgot 
them. Second, students sometimes cannot see that a given 
principle applies in the particular situation which has been pre- 
sented to them. And third, there are some students who are 
able to tell what did happen (explain) or can tell what will 
happen (predict), but they cannot tell why; and when they 
attempt explanation or justification of their decision, they come 
to grief for many reasons. 

An analysis of written papers showed that students often 
give false reasons, and even more frequently they give irrele- 
vant reasons for their predictions. Quite frequently, they assume 
what they were supposed to prove, and it is common to find 
them relying upon weak or even spurious authorities. They 
frequently use rather poor analogies, and misconceptions of 
truth, including superstitions, are to be found among their 


* Dewey, John. Human Nature and Conduct. New York: Henry Holt and Company, 
1922. pp. 238-39. 

*In explaining what has happened, or in predicting what might happen, students 
make use of facts and definitions in addition to principles. For the purpose of this discus- 
sion, emphasis has been placed upon the word “principles.” 
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arguments. They often ridicule a position and assume that 
this is a defense of their own, and some students blithely 
justify their own convictions by arguing that any other conclu- 
sion is ridiculous. In some situations they are prone to advance 
teleological arguments for the activities of plants, and animals 
other than man, and sometimes they even attribute purpose to 
inanimate objects. More common, perhaps, than anything so 
far mentioned is the disposition of many students to give only 
one fact or principle when adequate explanation of the event 
calls for one or more additional comments. 

Still another type of error which is common may be de- 
scribed as inconsistencies in reasoning or confusing the impli- 
cations of a prediction or explanation. Students show this 
confusion by choosing a prediction or explanation and then sup- 
porting it with reasons which explain a different conclusion. 


Ks tests are commonly used to get evidence concerning 
the student’s ability to apply principles. The teacher 
knows what principles he is teaching, and he can describe some 
situations in which one or more of the principles operate, and 


ask the student to write out his prediction and his reasons for 
the prediction. This method is time-consuming for the student; 
it limits the number of principles to be tested and the number 
of situations in which any one principle is to be tested. The 
teacher often finds the essay difficult to score, and there is always 
the likelihood that other characteristics of the essay—the hand- 
writing, the neatness of the paper, the grammatical construction, 
and perhaps the mere length of the discussion—will distract his 
attention from scoring the particular ability being tested—the 
ability to apply facts, principles, and definitions in situations 
new to the students. Sometimes a teacher must infer what a 
student means, from what he has written, because of a lack of 
clarity of statement. All sorts of subjective factors enter into 
this kind of evaluation and make it probably less accurate and 
certainly more wasteful of time and energy than another re- 
cently developed method. 

While a number of procedures have been developed for 
objectively testing the ability of students to apply facts, prin- 
ciples, and definitions to situations which are new, this paper 
will deal with only one method, which was developed experi- 
mentally. Students who did well on the essay tests used in the 
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experiment also did well on the tests made according to the 
plan to be outlined. Students who did poorly in the essay exam- 
inations also did poorly on the objective examination. The 
correlation was consistently above .83. This rather high corre- 
lation suggests that the new method is a good index of the 
ability to apply facts, principles, and definitions in new situa- 
tions, in so far as pencil-and-paper responses are concerned. 
The reader must remember, of course, that while the short-cut 
or objective type of test will give with considerable accuracy 
the rank of the student in class according to his ability to apply 
principles, and while it may indicate some of his shortcomings, 
the test will not reveal what a particular student would actually 
write in his essay form. 


a procedures in devising an objective test for the evalua- 
tion of the ability to apply principles involve the following 
steps. The first is to select the facts, principles, and definitions 
to be tested. The teacher may select these from the materials 
currently being taught in the classroom or from subjects which 
have been studied by the student some time during his school 


career. Because of limitations of testing time, the teacher will 
perhaps want to select only the most important principles, facts, 
and definitions which have been taught. 

The second step consists in setting up some problem in 
which the principles, facts, and definitions apply. This problem 
is usually referred to as a situation. The setting in which the 
principles apply should be new to the student; that is, it has 
not been taught in the classroom. The situation should be 
so described that questions can be asked regarding what will or 
did happen. Naturally, the problems or situations ought to be 
interesting to students, ought to be vital in the sense that they 
are important in the lives of students, and ought to be within 
the range of student experience. 

In the third step, the teacher presents two or more plausible 
alternative answers. The student is directed to check those 
which are most consistent with the facts given and most reason- 
able in the light of whatever knowledge he may possess. 

Instead of asking the student to write out the reasons for 
his answers, the objective test should include a list of reasons 
from which the student can choose from the plausible ones 
presented those which he believes explain or support the con- 
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clusions. Formulating these reasons constitutes the fourth step 
in the procedure of test-making. Experience has shown that 
the teacher can save time by thinking of the most reasonable 
conclusions and then writing out all the reasons for believing 
the conclusions to be best. The teacher may then take each of 
the other conclusions and do the same for it. When she has 
supplied reasons for each conclusion, they should all be put 
together in a scrambled order. The test is now complete and 
ready to give to the students.° 


HE procedure applied to an actual situation is shown in the 

following description. The teacher in question was con- 
cerned with materials relating to health and particularly those 
dealing with the growth of bacteria. She wanted the children 
to understand that the souring of milk is the result of the 
growth and life processes of bacteria which are retarded by low 
temperatures. She wrote out the problem in this fashion: 

In warm weather [a woman] who does not have a refrigerator 
sometimes wraps her bottle of milk in a wet towel and places it where 
there is a good circulation of air. Assuming that the temperature is high 
and that the humidity is low, would a bottle of milk, wrapped in a wet 
towel and placed where the air was circulating, stay sweet as long as a 
similar bottle of milk without a wet towel? * 


In thinking of the conclusions which students might come to 
after reading this problem, the teacher listed these three: 
The bottle wrapped with the wet towel would stay sweet: 


a. longer than the other bottle . . . .... +. (~*~ dea 
b. not so long as the other bottle . . . . ( jb 
c. the same length of time—the wet towel would make no 

eee ee ee ee ee ee ee ee ee ee ee ee 


Here again, the teacher will find it a rather easy matter to give 
some reasons for each of the alternatives. The teacher will 
work out the problem herself and would probably check as the 
right answer—“a. The bottle wrapped with the wet towel 
would stay sweet longer than the other bottle.” She might give 
as her reasons for that judgment: 

1. Evaporation is accompanied by an absorption of heat. 

2. Bacteria do not grow so rapidly when temperatures are kept low. 


® See pages 95-98 for interpretation of results. 
“In the most reasonable solution of this problem, the teacher realized that a principle 
involving evaporation was necessary in the explanation, but her class had already been 
taught that evaporation is accompanied by an absorption of heat. 
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3. The souring of milk is the result of the growth and life processes 
of bacteria. 
The teacher might now proceed toward making some reasons 
for conclusion c: 
4. A wet towel could not interfere with the growth of bacteria in 
the milk. 
5. Wrapping the bottle with a wet towel is merely an old super- 
stition. 
In writing up some reasons for conclusion 4, she might write: 
6. The water in the towel absorbs heat and makes the milk stay 
warm longer. 
7. Wrapping a bottle of milk makes it warmer in the same way that 
people put on extra wraps to keep warm. 


With all these reasons developed, the next step is to rearrange 
them in a scrambled order, and the test items which follow the 
description of the situation would be as follows: 


The bottle wrapped with the wet towel would stay sweet: 


a. longer than without the wet towel . . . . «. . ( % da’ 

5. not as long as without the wet towel . . ( jd 

c. the same length of time—the wet towel woudl ale no 
difference . . cat 


Choose the statements which you would u use to explain or support your 
conclusion: 


d. The souring of milk is the result of the growth and life 


processes of bacteria . . <= (age 
e. A wet towel could not interfere with the growth of bac- 

teria in the milk . . . Je 
f. The water in the towel absorbs heat and makes the milk 

stay warm longer . . So 
g- Wrapping the bottle with ; a wet towel 3 is merely 1 an old 

superstition . . « & = 
h. Evaporation is accompanied by an | absorption of heat . ee 
i. Bacteria do not grow so rapidly when on are 

kept low. . iw. «ee 


j. Wrapping a bottle of milk tmakes it warmer in the same 
way that people put on extra wrapsto keep warm . (_ )j 


N OUTLINE has been presented for the making of test exer- 
A cises in which the student has the opportunity to indicate 
which of several reasons presented are consistent with the choice 
which he made among the conclusions. Some teachers have 


° The letter x indicates the scoring key developed by the teachers. 
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varied the procedure by including among the reasons many 
fallacious reasons like those which students often give in sup- 
port of their decisions in school. The correct application of 
scientific facts, definitions, or principles to a new situation is 
probably not a simple, unitary ability, but a complex of mental 
habits, attitudes, understandings, skills, and inhibitions which 
brings into play almost all the elements of “scientific method.” 
As an appraisal of scientific thinking, therefore, a test of the 
application of principles to new situations may well include 
among its incorrect responses some characteristic violations of 
the spirit and method of scientific inquiry. 

In defending a decision to which they have come, some stu- 
dents, instead of giving good reasons, merely assume what they 
are trying to prove. Let us assume that a student has checked 
the right conclusion, and that we want to discover whether or 
not he will give substantial reasons for his conclusion or whether 
he will jump to some statement which merely assumes what 
he has already decided to be the answer. We might well add a 
statement which is in effect a repetition of the true answer and 
which might begin with any of the following phrases: everyone 
knows, experience proves, it is generally agreed, everyday liv- 
ing shows, it is naturally followed, it is ridiculous or absurd to 
believe anything different, it is only common sense to say that. 

Sometimes in support of their decisions, students will offer 
analogy as evidence. Not infrequently, analogies are useful as 
illustrations, but quite often they are used where the analogies 
do not apply exactly and when they do not serve as evidence 
for proof at all. Teachers who want to make use of this form 
of reasoning error in developing statements might begin the 
statements with such phrases as the following: just as, accord- 
ingly, since, like, even as, much the same as, in like fashion, 
similar to, corresponding to, from the fact that . . . we may 
conclude, bearing in mind that, and remembering that. Stu- 
dents often use the irrelevant answer. Teachers will not find it 
difficult at all to supply a number of statements which are 
slightly irrelevant to the correct answer. 

Quite frequently there are false statements included in the 
essays which students write when they defend a particular 
answer that they have written out. It is not difficult for teachers 
to devise statements which are made up of false evidence of 
the kinds commonly supplied by students. 
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In some situations there are common misconceptions which 
apply. Some teachers are more skillful than others in their 
discovery of these most common misconceptions; but it must 
be remembered that they do not apply in every situation, and 
should be used only where they have application. Misconcep- 
tions may be described by saying that they are ideas commonly 
held which are not supported by evidence. Teachers who want 
to extend their knowledge of the common misconceptions in 
science may depend upon contact with people who have not had 
high-school or college training, may get some of it from the 
literature which they read, but will have to depend largely 
upon knowing the kinds of misconceptions and errors which 
students make. 

A sixth kind of error which students make in justifying their 
conclusions is that of relying upon false or spurious authority. 
Teachers who are anxious to get at the information of their stu- 
dents who make this error may well develop statements which 
begin with: advertisements in the best magazines say; science 
says; various famous people say; certain books say; in a maga- 
zine article it said; well-known industrial leaders say; research 
organizations say; father, mother, teacher said; and over the 
radio, a speaker said. 

Sometimes the situation is one dealing with living things 
other than man—or sometimes with inanimate nature—and 
we are able to see whether or not students select teleological or 
anthropomorphic statements. These might be written so that 
they will have the following phrases in them: animals do certain 
things—in order to; so that; for the purpose of; for (in the 
sense of desire); to (in the sense of desire); because of need, 
desires, wants, wishes, likes, fears; divinity, inscrutable purpose 
(mystical) ; because nature wants. 

Sometimes students try to defend a particular point which 
they have taken by ridiculing an opposing viewpoint. By de- 
veloping lists of such statements in connection with a particular 
situation, teachers may find out who of their students are most 
likely to ‘fall into this kind of error. It is possible to write some 
statements which would say, in effect, that any other conclusion 
but the correct one is sheer nonsense; that it is absurd to believe 
anything else; that only untrained students would choose an- 
other; that every sensible person would choose this one. 

If these various types of statements are made and are appro- 
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priate to the situation for which they are made, the teacher may 
be able to get various kinds of evidence from the answers which 


the students have given.° One exercise constructed according 
to these procedures follows: 


What would happen to the weight of a piece of meat if it were 
placed uncovered in an electric refrigerator? 

Directions: Choose the conclusion which you believe is most consistent 

with the facts given and most reasonable in the light of your knowledge. 

Conclusions :* 

nih. It will gain weight. 

——B. It will lose weight. 

C. It will remain at the same weight. 

Directions: From the following list, choose the reasons you would use to 

explain or support your conclusion. 

Reasons: 

1. Water vapor is given off by the meat at any temperature. 
(Correct reason) 

2. The temperature in the electric refrigerator is lower than 
room temperature. (Irrelevant reason) 

——3. It is foolish to believe that food, when placed in a refrigerator 
which is almost airtight, would lose weight. (Ridicule) 

—+4. Just as a block of ice weighs less than an identical volume of 
water, so will meat weigh less after it is placed uncovered in 
an electric refrigerator. (False analogy) 

—5. The expansion coil is the coldest part of an electric refriger- 
ator. (Correct reason) 

—6. Manufacturers of electric refrigerators claim that food when 
properly stored in their refrigerators will neither lose nor gain 
weight. (False authority) 

——7. The increased weight is caused by evaporation. (‘Technically 
false) 

——8. Water vapor tends to condense on the coldest part of the re- 
frigerator. (Correct reason) 


Reasons inconsistent with various conclusions were: with A, 1, 4, 


5, 6, 8; with B, 3, 6, 7; with C, 1, 4, 5, 7, 8. 


_ student who makes errors in the test exercises some- 
times does so because of his inability to apply facts, prin- 
ciples, and definitions to new situations. Sometimes the student 
does not have the necessary information; in other words, he 


° Quite obviously, there are many other types of errors in reasoning which teachers 
may want to include in the tests which they make. These are offered only as illustrations 
of a few of the commonest types. 

* The letter x indicates the scoring key developed by the teachers. 
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may not know the facts, principles, and definitions which apply. 
If a teacher feels that some of her students may not have the 
necessary information, it may be wise to give to the class a list 
of facts, principles, and definitions which have been taught and 
to include with them a list of facts, principles, and definitions 
which are not true and ask the students to check those which 
they know to be true. The use of an information test may 
eliminate this possibility of misinterpreting the test results. 

Generally the teacher wants this ability described in three 
ways. She wants to know with what facility the student can 
select a conclusion or make a prediction which is most consistent 
with the facts given. The student reveals this ability when he 
is presented with problem situations and asked to predict what 
will happen or to explain what did happen under certain cir- 
cumstances. The teacher wants to know with what facility the 
student can present reasons which support or explain his conclu- 
sions. This information gives the teacher some knowledge of 
the comprehensiveness of the student’s reasoning; it is revealed 
by the number of sound reasons chosen by a student to support 
or explain his conclusions. In this same connection, a teacher 
wants to know what percentage of the student’s reasons are 
correct. This is easily determined by counting the total number 
of reasons and then tabulating the number of those correct. 

Finally, the teacher wants to know the types of errors in 
thinking which are made by a student, so that a plan for remed- 
ial instruction may be set up. If these “thinking errors” have 
been part of the design in the construction of the tests, it be- 
comes a relatively simple matter to report the frequency of the 
different types of errors for each student. 


— illustrative descriptions of the test results of different 
students are given in Table I. The scores made by eleven 
students are grouped here for convenience of reference. The 
reader will note that the first three students made accurate 
predictions in every test exercise in the examination. At the 
same time, however, note how different the scores are when 
an analysis is made of their reasons. John Gregg chose 80 per 
cent of the available correct reasons to support his conclusions; 
among his incorrect reasons he was more prone to assume the 
conclusions and to use false analogies as reasons. In a few 
cases, he chose reasons which actually supported conclusions 
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different from those of his own selection. Contrast his record 
with that of Mary Pickering. All of her predictions were cor- 
rect, and yet she chose only 46 per cent of the available correct 
reasons to support her conclusions. Her errors were concen- 
trated in assuming what she was trying to prove; she made more 
errors of this type than any one in the group of forty-five stu- 
dents. She also relied frequently upon false authorities and 
upon false analogies and chose many true but irrelevant reasons. 
Notwithstanding these frequent errors, she showed no incon- 


TABLE I 


PERCENTAGES OF DESIRABLE AND UNDESIRABLE RESPONSES 
APPLICATION OF PRINCIPLES IN SCIENCE 





| Desirable | Undesirable 
Name ————<$_—_ 


A | |e | F!1cG|H 


(1) (z)_ | G@) |_ (6) | (7) _|_ (8) 


Gregg, John 100 80 | 30 | 36 | «© 13 
Pickering, Mary 100 (| 40 | go | 50 | Oo 
Haskett, Edward 100 | 10 10 ° oO 


Drexel, Irving go Oo }-@ | «© ° ° 
Conaway, David go | 30 go 60 13 
Davis, Edward go 3 20 be) ° o 
Bigler, Ruth a ie 50 z0 | 10 13 


Beatty, Bernadine .... 50 | 70 80 70 
Sternbaum, Violet .... 50 3 20 40 | 20 
Wallace, Miriam 40 «| 20 40 30 


Slyh, Horace 40 ° ° 


70 | 34 | 12 Il 20 | 10 | Oo 8 
| 30-100 | 8-80 | 0-34 | 0-99 | 0-90 | 0-70 | 0-50 | 0-36 

















Key to Column Headings: 
A—Correct predictions F—Assuming conclusion 
B—Correct reasons G—False authority 
C—Technically false reasons H—Ridicule 
D—True but irrelevant reasons I—Inconsistency 
E—False analogy as reasons 


sistencies whatsoever. Her reasons, no matter how specious, 
always tended to support the conclusions which she selected. 
The case of Edward Haskett is still different. He, too, has a 
perfect score in making predictions and, like Mary, he has only 
46 per cent of the correct reasons to support his conclusions. 
This boy, however, is consistent in his reasoning and makes few 
errors. It is evident that his relatively low score in Column 3 
(correct reasons) is attributable to the fact that he did not 
choose many. This may be due to caution, to a lack of knowl- 
edge, or to an inability to see that the principles applied. 
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The next group of students had high scores in choosing 
conclusions which were most consistent with the facts given. 
Note again, however, how the patterns of achievement differ 
with respect to all columns but the first. Irving Drexel’s 
achievement includes none of the types of errors which were 
tabulated. David Conaway has made many errors, and these 
are distributed over many different types. Edward Davis and 
Ruth Bigler are distinguished from the other two in Column 3, 
which represents the percentage of correct reasons chosen to 
support their conclusions. These two students differ from each 
other when the remaining columns are studied. 

The four students in the lowest group show similar wide 
variations with respect to type of achievement. Notice that 
Bernadine Beatty has only 50 per cent of the correct conclusions 
and yet has chosen 70 per cent of the available correct reasons. 
In this latter respect, she stands second highest in the class; but 
note the many errors and the many types of errors represented 
in her reasoning. Compare her scores with those of Horace 
Slyh who has almost as many correct predictions to his credit, 
many fewer correct reasons, and practically no errors in reason- 
ing. It is interesting to note, too, that these four students have 
higher scores in Column 4 than do the others. They evidently 
have chosen a number of technically false reasons to support 
their conclusions. This points to a lack of knowledge of the 
correct principles and where the scores in Column 3 are low, 
points also to a lack of ability in applying principles. 


[7 a period of several years, studies were made of the 
relations between the results of a test of application of 
principles and the results of a test of the interpretation of data.* 
Sixteen separate studies were reported; the number of students 
in each investigation varied from 22 to 684. The correlation 
coefficients, corrected for attenuation, ranged from .33 to .54. 
These findings are substantial justification of the judgment that 
we must appraise each of these abilities separately; achievement 
in one is not always accompanied by achievement in the other. 

The relationship between scores on a test of the application 
of principles and scores made on an information test was found 
to be similar to those between the results of a test of application 


* Judd, Charles H. Education as Cultivation of the Higher Mental Processes. New 
York: Macmillan Company, 1936. pp. 12-17. The coefficients reported with one excep- 
tion were gathered in studies made by the Tests Division, Bureau of Educational Research, 
Ohio State University. 
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of principles and a test of interpretation of data. Sixteen studies 
were reported with coefficients of correlation ranging from .31 
to .§8. One would naturally expect a somewhat higher rela- 
tionship here because the ability to apply principles is dependent 
upon knowledge of them. 

Frutchey reported the largest percentage of retention was 
found in the ability to apply principles; in his study the other 
variables measured included the selection of facts, terminology, 
symbols, formulas, valence, and balancing equations. He states 
that his study substantiates other studies of retention of achieve- 
ment involving objectives other than information and adds that 
retention was greatest in the more general types of behavior.” 
Tyler reports that during a fifteen-month period, there was no 
loss in the ability of students to apply important zodlogical 
principles to new situations or to interpret data obtained from 
experiments.”” 


Measuring the Interpretation of Data 


\ N J HEN teachers talk about “the interpretation of data,” 
they usually think that the word data means evidence 

or “facts” presented in running comments, or in tables, 

graphs, maps, cartoons, pictures, or other materials of a pub- 
lished sort. They also include nonpublished materials such as 
facts presented in motion pictures or in radio broadcasts, as 
well as actual firsthand experiences, both in and out of school. 
These data are usually thought of as referring to areas 
which are related to the curriculum of the school. The compila- 
tion of lists of areas has been common. A list arranged by 
teachers in Greeley, Colorado, contains 28 major types of 
problems. The “Science Report of the Progressive Education 
Association” also contains many. Mr. O. I. Frederick, of Mis- 
sissippi, reports a collation of many such lists, and it contains 9 
large problem areas. The following list of such major areas 
has been selected from certain unpublished reports to which the 
writer has access: housing; crime; recreation; racial relations; 
nationalism, imperialism, and war; labor organization and labor 


* “Retention in High-School Chemistry,” Epucationat Researcu Butietin, XVI 
(February 17, 1937); PP. 34-37: 

1° “Permanence of Learning,” Constructing Achievement Tests. Columbus, Ohio: 
Bureau of Educational Research, Ohio State University, 1934. p. 77. 
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relations; family problems; religion; intelligent consumption; 
taxation; money and credit; and transportation and communi- 
cation. 

Teachers frequently in their discussions of the “interpreta- 
tion of data” give it three meanings. The first is, interpretation 
may be a summary of the data. This summary may represent 
the description of a trend in the materials; and in some cases 
this trend will be shown by part of the data which may be con- 
trasted with the remaining portion of the data. Or again, sig- 
nificant points within the data which may be extremes or which 
may represent peculiar deviations from the general trend may 
be distinguished; or the data may be manipulated in such a fash- 
ion that one or more inferences can be drawn which are not 
readily apparent in the facts as they are presented. In all of 
these cases, of course, the interpretation is always one which is 
supported by the presented facts; it is not qualified as “probably 
true.” In all its varieties, this meaning of interpretation em- 
braces both the making of inferences and actual reading or 
re-reading of the facts given. 

A second common meaning of interpretation is associated 
with a less prosaic activity and is focused upon making qualified 
inferences from the facts given. In general, this type of inter- 
pretation can be accomplished successfully only where trends 
are revealed in the data and the inferences involved consist of 
prudent extrapolation or interpolation of the data. A student 
who does this type of interpreting well employs his imagination 
in predicting conclusions which are somewhat beyond the facts 
but which are in agreement with a definitely established trend. 
In making this type of interpretation, the student always quali- 
fies it with the phrase “probably true.” Even where this type of 
inference is qualified, the student is careful not to go far beyond 
the observed facts and limits his predictions to rather narrow 
ranges. What has been said about extrapolation, or going be- 
yond the facts intelligently, is also true of interpolation, that 
is, going between the observed data and risking an inference 
concerning an omitted point in a set of data which reveals an 
established trend. Conversely, students who do well in this 
type of interpreting are quite apt to designate as “probably 
false” an observation or inference which contradicts an estab- 
lished trend. 

The third most common meaning attached to the word 
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interpretation is related to the recognition of the limitations 
of data. In the preceding paragraph, the emphasis was upon 
squeezing out of the facts not only that which was indisputably 
true but additional inferences which could be designated as 
“probably true.” Implied in that purpose was the aim to get 
as much out of the data as possible. That concept is to be 
contrasted with the present third meaning of the word interpre- 
tation. In recognizing the limitations of data, the student who 
interprets well will see that many interpretations must be re- 
garded as almost completely uncertain because the facts are 
insufficient to support these interpretations even qualifiedly. 

Teachers who strive to develop in their pupils the abil- 
ity to interpret data emphasize in their teaching all three of 
these aspects. Students are guided to make unqualified inter- 
pretations which are rather fully supported by the available 
evidence. Students are shown how to make interpretations 
which are somewhat beyond the facts and which may yet be 
thought to be in the realm of probable truth. They are also 
shown how to recognize the limitations of the available evidence 
and to recognize that certain types of interpretations may not 
even be thought of as qualifiedly supported; teachers suggest to 
students that the last type may be taken as a hypothesis and 
other data sought which may have relevance to the point. 

The data presented to students most frequently are new 
in the sense that this particular organization of the facts has 
not been discussed previously. Interpretation is thought of as 
a developing ability. Experimental evidence shows that this 
ability does not transfer perfectly from one area to another. 


pe following outlined procedure is suggested for the 
actual construction of a test to appraise the student’s ability 
to interpret data.’ It has been followed by many teachers with 
successful results, and it may be modified to suit individual 
and local needs. It is offered only as a guide and should be 
varied to meet the purposes of teachers. 


1. Choose some data which students are to interpret. 
2. Make a series of interpretations from these data, patterned after the 
three major categories mentioned here. These will take the form of 
1 Obviously, it would be good practice to have data collected by students in their 
attack on the solution of problems which are vital to them. Where this is not the pro- 


cedure, it would certainly be wise to have data chosen which have significance in the lives 
of the students. 
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a. Statements which are rather fully supported by the data. 

b. Others which may be thought of as only “probably true” and which 
are partially supported by the data. 

c. Statements which represent interpretations which are not supported 
by the data and which must be classified as “uncertain” because 
they are based upon insufficient evidence. 

d. Statements which contradict some established trends within the 
data and may therefore be classified as “probably false.” 

e. Statements which are actually contradicted by the evidence and 
may therefore be labeled as “false.” 


Let us follow through this procedure with an example. 
Let us suppose that the data chosen for the test devoted to the 
interpretation of data comprised percentages regarding the per- 
sons gainfully employed in the United States taken from the 
book, Recent Social Trends. We shall follow the outline and 
make statements which are true, some which are probably true, 
some which are based on insufficient evidence, some which are 
probably false, and some which are false. Then we shall pre- 
pare the materials for the students and check the directions. 
In arranging the items in the test, we shall not put them in 
logical order. Here are the statements as they are presented 
in the test preceded by the data to be interpreted: 


PERCENTAGE DisTRIBUTION OF GAINFULLY OccUPIED PERSONS IN THE 
Unitrep States SIXTEEN YEARS OF AGE AND OVER, 1870-1930 





| j 
| 





Occupation Group | 1870 | 1880 | 1890 | 1900 | 1910 | 1920 | 1930 

(1) | (2) |_ G) | G@) | Gs) | (6) | (7) _ | (8) 

MaMa os i5 e des .. | S28] 48a] gna | 35.9! 30.3 | 25.8) 21.3 
PO eosin Ck ere aS She Se 1.5 1.6 1.8 | 2.3 2.6 2.7 2.0 
WEOMUESCONIIN® oo 6552 Boa 303 22.0] 24.8! 26.3 27.5 28.6 30.5 28.6 
Trade and transportation...... ~. | Qel 10.7 13.6 16.3 17.4 18.0 | 20.7 
ee ; Pi 2.0 2.5 2.8 4.6 7a 8.2 
Domestic and personal service. . . 9-6 8.8 9.7 10.0 10.6 8.8 11.3 
PUGHe GOIWIGE 5 x sche eke ei el 6 | a ‘9| 1.0 I.1 1.6 1.4 
Professional service ........... | 29 3-3 4.0} 4.4] 4.8 5-4 6.5 
TRON ae Sb ortind nema ns 100.0 | 100.0 | 100.0 | 100.0 | 100.0 | 100.0 | 100.0 


Statements— 

— 1. There has been a persistent decline in the percentage of gainfully 
occupied persons in the United States, sixteen years of age and 
over, in the field of agriculture from 1870 to 1930. 

— 2. The number of ministers has increased from 1870 to 1930. 

— 3.In 1910, the largest percentage of occupied persons was to be 

found in manufacturing. 
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4.In Great Britain, shifts in occupations of people gainfully em- 
ployed have taken place which are similar to those in the United 
States. 
In 1930 the percentage of gainfully occupied persons was greater 
than in 1850 for all occupations excepting agriculture. 

. We shall be faced with a shortage of food in 1960 because of the 
percentage of decrease in the occupation of agriculture. 

. The percentage of increase of persons engaged in trade and trans- 
portation from 1870 to 1930 is greater than any other occupa- 
tional group. 

In 1940 the percentage of gainfully occupied persons in trade and 
transportation will be somewhat more than in 1930. 

There has been a steady increase in the percentage of gainfully 
occupied persons in all of the occupation groups. 

. More people were engaged in farming in 1930 than in 1870. 

All occupation groups except agriculture increased in the percent- 
age of gainfully occupied persons from 1870 to 1930. 

—12. The increased percentage of persons employed in manufacturing 
has been of great benefit to our nation. 

—13. In 1940 the percentage of gainfully occupied persons in trade and 
transportation will be less than in 1930. 

—14. From 1920 to 1930 there was a decrease in the quantity of agri- 
cultural products produced in the United States. 

—15. The development of farm machinery is one of the chief reasons for 
the reduction of the relative percentage of people engaged in 
agriculture. 

—16. In 1940 the percentage of persons engaged in agriculture will be 
more than in 1930. 


or the purpose of this article, let us assume that we have 

made ten or twelve such exercises, all of them relating to 
data which teachers believe are important. Moreover, let us 
assume that the suggested interpretations have been made ac- 
cording to the five types described and that if we counted the 
different interpretations the statements would take approxi- 
mately the following distribution: 

Per Cent 
PMR RIG 5 6625454 5i0 3S BIA SIS OS Warn Sieuee eae 20 


Uncertain 
Probably false 


Let us assume further that in a complete test approximately 
one hundred such interpretations have been presented to stu- 
dents, that they have taken the test, and that we have scored 
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the papers. How can we describe students’ achievement which 
will meet the needs of teachers? We might set up a chart and 
tabulate all of the students’ responses in the manner given in 
Table I. 

This is easily done because we have the student’s paper 
before us, and we have a key which has been worked out with 
the teacher. By glancing at the chart, we can tell immediately 
how many of the true statements the student recognized accu- 
rately. This will be in the little square labeled a. If we count 
all of the tally marks in the little squares labeled a, g, m, s, and 
y, we shall have the total number of statements which a student 
recognized accurately. When we remember that there were 


TABLE I 


SumMarRY SHEET—ASPECTs OF THINKING* 


| This student marked these statements as: 
Number of statements in 


Prob- Insuffi- | Prob- | 


this test which SHOULD True ably cient | ably | False Omit- 
have been marked as: Tae ase Baloo ted 
| (1) (2) | G) | (4) | Gs) | 
True were 10 (1) la b | ¢ d | e | 
Probably true were 20 (2) | f g h | é | 7 | 
Insufficient data were 40 (3) | & l m n | 0 
Probably false were 20 (4) | p r | s | ¢ 
False were 10 (5) | u |v w | x | y | 


* This test can be modified so that it can be used for machine scoring. Directions 
have been worked out which make it possible to use the new scoring machine, and it is 
possible now to score approximately 100 papers of this kind an hour and to record seven 
different results which are a part of this single test. 


Total Omitted 


100 statements in the entire test, we see immediately that the 
total number which the student had correct represents the per- 
centage of all the possible correct responses which were pre- 
sented in this test. 

If we want to know how well the student recognized those 
“probably true” or “probably false,” we can refer to the little 
squares marked g and s. If we add these two amounts together 
and divide by 40 (there were 40 such statements altogether in 
the test), we shall then know the percentage of the qualified 
interpretations which the student had correct. 

If we want to know how well the student was able to recog- 
nize the limitations of the data presented, we can focus our 
attention upon the little square labeled m. There were 40 such 
statements in the test, and we have only to divide the number 
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which he recognized correctly by 40 to get the percentage of 
correct responses in this category. 

If the teacher, by a study of this chart, would like to know 
how frequently a student went beyond the facts, she might turn 
her attention to all of the squares included in f, &, 1, m, 0, and ¢. 
We see that these represent behavior on the part of the student 
in which he took statements that were probably true and called 
them true, or took statements that should have been designated 
as probably false and called them false, or he took statements 
which really were based on insufficient evidence and called them 
probably true or true—or designated them as probably false or 
false. All in all, there were 80 opportunities for the student to 
do this, and here again it would be an easy matter to calculate 
the percentage of the times a student went beyond the facts. 

If the teacher is interested in knowing the tendency or the 
disposition of a student to go beyond the facts in the direction 
of ascribing more truth to the data than is warranted, she can 
direct her studies toward the squares labeled f, &, and /; the 
tallies recorded in these spaces all point to a student’s tendency 
to ascribe more truth to statements than is warranted. 

If the teacher is interested in knowing the tendency of a 
student to reject a statement as “more false” than the facts 
would warrant, she might focus her attention upon the squares 
labeled n, o, and t. Here again, the same 60 statements must 
be counted as opportunities, and, as in the other cases, it is an 
easy matter to determine a percentage score. 

To get some measure of the cruder errors in judgments 
which students are likely to make, a teacher needs only to study 
the spaces labeled d, e, i, and 7, and those labeled 9, g, u, and v. 
Tally marks in these spaces indicate that a student is confusing 
truth and falsity. 

Table II reports the scores made by four students of a 
twelfth-grade class in a test of this sort.” The first student in 
the list, Ross Sunny, represents the highest achievement in 
almost all aspects of the test. In Column 2, we notice that his 
general accuracy in interpreting statements is represented by a 
score of 72 per cent, and this is the highest score in the group. 
We note that he had 60 per cent of the probably true and prob- 
ably false interpretations correct—again the highest score in 


* More information concerning the achievement of a great majority of students in 
the Lakeshore School on this particular test is recorded in the EpucaTionac ResEarcu 
Buttetin, for March, 1938, on page 62. 
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the group. He seems to be able to recognize the limitations of 
the data (see Column 4) for he has recognized 77 per cent of 
the opportunities wherein data are insufficient to make judg- 
ments. In the negative characteristics revealed by the test, his 
scores are low. He went beyond the facts only 19 per cent of 
the time. In ascribing greater truth or greater falsity to the 
data than was warranted (see Columns 6 and 7), his scores were 
low, and he made almost no crude errors in judgment, which 
include confusing true statements with false ones or “probably 
true” statements with those which are “probably false.” There 


TABLE Il 


PERCENTAGES OF DEsIRABLE AND UNDESIRABLE INTERPRETATIONS OF 
SociaL Data GIvEN By STUDENTs oF GraDE XII 
(All Numbers Are Percentages of Total Opportunities in the Test) 








Student’s Name 
(1) 


Sunny, Ross 

Hyle, Leon 
Diederly, Priscilla. . 
Taber, Helen 


mwnn 
N Of N 


Group median .... | 3 63 41 32 14 | 
8-28 | 2 


Group range... 2 7 12-40 | 


Key to Column Headings: 

A—General accuracy with respect to all statements in the test 

B—General accuracy with probably true and probably false statements 

C—General accuracy with statements based on insufficient data 

D—Going beyond the data 

E—Going beyond the data in the direction of ascribing more truth than is warranted 

F—Going beyond the data by ascribing greater falsity than the data warrant 

G—Confusing probably true with probably false, or true with false, or vice versa, crude 
errors in judgment 


is a consistent performance in all aspects of the test, significantly 
high and revealing a clear-cut, consistent application of the 
principles of interpretation. 

The scores of the second student, Leon Hyle, are some- 
what less consistent. In general accuracy in interpreting mate- 
rials, he stands at about the median of the class. This is true 
also of the ability shown in Column 3, which represents his 
accuracy in recognizing statements that are probably true and 
probably false. In Column 4, however, we note how strong he 
is in recognizing the limitations of the data, in grasping the fact 
that the evidence is insufficient to make a judgment. On the 
negative side of the table we note that he went beyond the facts 
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(Column 5) much less than was typical of the group, and yet 
we note that he made many more crude errors in judgment than 
was typical of his group (see Column 6). Leon would evi- 
dently profit much from guidance with respect to this ability, 
and it could most profitably be directed toward Column 3 and 
Column 8. 

Priscilla Diederly, the third student in the list, shows quite 
a different pattern of achievement. Here we have a student 
who recognizes half of the statements which were in the test 
and judges them at their proper worth. She seems to do rather 
well in handling the qualified interpretations and has a rather 
high percentage of those correct compared to the median of the 
group (Column 3). When we look at the results of her 
achievement in Column 4, however, we note at once that she 
has the lowest score in the group with respect to recognizing 
the limitations of the available facts. When this is compared 
with her score in Column 5, going beyond the data, we note that 
here, more than anyone else in her group, she tended to make 
judgments without due regard to the evidence. She is lowest 
with respect to recognizing the limitations of the data and high- 
est with respect to going beyond the facts. She was frequently 
“oullible” in that 38 per cent of her responses ascribed greater 
truth to the interpretations than the facts would warrant. Not- 
withstanding this performance of Priscilla’s, we note that only 
10 per cent of the time did she make crude errors in judgment, 
ranking only second to Ross Sunny in this respect. The teacher 
could profitably direct Priscilla’s efforts toward studying data 
more carefully in the hope that she would come to recognize 
their real worth. 

If we turn to Helen Taber, we see a still different pattern 
of responses. In general accuracy, she has the lowest score in 
her group. In her ability to handle statements which must be 
qualified as probably true or probably false, she is again almost 
at the bottom of the class. When it comes to recognizing the 
limitations of the data, however, Helen shows almost average 
ability. In Column 5 we get some estimate of her tendency to 
go beyond the data, and we see again that her score represents 
about the median of the group. This is also true of her score 
in Column 6, which is associated with her disposition to ascribe 
greater truth to the interpretations than the facts would war- 
rant. In Column 7, however, we see again a departure from the 





APRIL 13, 1938 107 


group pattern. Here Helen has shown the highest disposition 
of any in her group to reject as “more false” than the facts 
would warrant, many statements which should have been 
marked either as uncertain or as probably true. In Column 8, 
we note that there are many crude errors in judgment. In fact, 
she has made more of them than any other person in the group 
of which she is a member. In nearly all aspects of the abilities 
represented in this test, Helen’s achievement is low and teach- 
ing guidance would probably best take the form of general aid 
in every area.” 

The student’s responses were compared to the median of 
the group and to the range of scores because the writer believes 
that the first use which should be made of students’ achieve- 
ment on a test of this sort involves comparing accomplishment 
of the present date with that of a previous time so that some 
estimate of growth may be made. With respect to the present 
test, however, these students had not taken it before. 


BVIOUSLY, there is no single way to make a test of inter- 
pretation of data. It should meet the purposes of the 
teacher, and any suggested form should be modified in order 


that those purposes can be met. The test described in this 
article has been modified in a number of ways to meet the needs 
of different teachers. Some teachers have insisted that after a 
student has made a decision about an interpretation, he should 
be required to give the reasons which support his judgment. 
Other teachers believe that the fivefold discrimination is alto- 
gether too fine to expect of students in the high school, and 
they have developed tests of their own which require the stu- 
dents to make only three distinctions: to recognize statements 
which are supported by the data, to recognize statements which 
are contradicted by the data, and to recognize statements which 
are neither supported nor contradicted. Undoubtedly, there are 
many other modifications which have been made and still many 
others which will be made in the future, depending upon the 
purposes of the teacher. 


® Before deciding upon what to do, the teacher would surely consult other sources of 
evidence. The teacher would want to know the results from other tests. When she has all 
of the significant information assembled, she will be in a better position to suggest next 
steps to Helen. 





Evaluating Some Aspects of Proof 


ATHEMATICS teachers have been contending for many 
M years that an important objective of their teaching is 
the development in their students of what is loosely 
called “thinking.” Much has been written by teachers about 
their efforts to promote growth in thinking in mathematics 
courses. A significant contribution in this field has been made 
by Harold P. Fawcett, a teacher of mathematics in the upper 
division at the Ohio State University High School. In 1936 he 
wrote a paper which described in summary form the major ob- 
jectives of his teaching efforts and some examples illustrating 
the content and procedure of his teaching. This bulletin was 
distributed by the Progressive Education Association, and efforts 
to reconstruct the mathematics courses were intensified in many 
places. He tried to get his pupils to understand the place of defini- 
tions and assumptions in any discussion and the relation between 
the conclusions reached in any area of thought and the defini- 
tions and assumptions on which those conclusions might depend. 
When this general objective was restated in terms of student 
behavior, the following characteristics were considered of great 
importance: students who are growing in this understanding 
will select the significant words and phrases in any statement 
which is important to them and ask that they be carefully de- 
fined; they will require evidence in support of any conclusion 
that they are pressed to accept; they will analyze that evidence 
and distinguish fact from assumption; they will recognize both 
the stated and unstated assumptions essential to any conclusion; 
they will evaluate the underlying assumptions, accepting some 
and rejecting others, depending upon the circumstances; they 
will evaluate the argument, accepting or rejecting or doubting 
the conclusion; they will constantly re-examine the assumptions 
which are behind their beliefs and which guide their actions. 
Some evidence of a pupil’s understanding of the nature of 
proof can be gathered by utilizing techniques already described 
in this issue—the interpretation of data and the application of 
principles. A student who continually goes beyond the data 
may not be alert to the evidence which is required to support 
a conclusion. We can get some measure of the ability of stu- 
dents to select significant words and phrases which need defi- 
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nition by utilizing the techniques described for testing the 
ability to apply principles. We need only to find situations in 
which words and phrases need definition, present them to stu- 
dents, ask for a judgment, and suggest that these should be 
supported with reasons. For example, we might present to 
students situations like the following: 


In many cities there are two forms of street transportation in com- 
mon use. One of these is the usual streetcar, which runs on a rigid 
track and operates by electric power from an overhead trolley. The 
other is the bus or motor coach which runs on rubber tires, is free to 
move about in the street, and is powered by a gasoline engine. 

Recently, at Columbus, Ohio, the transportation company put in 
service a new type of vehicle to haul passengers. The new vehicle has 
rubber tires, and can move about in the street, but is operated by electric 
power from an overhead trolley. 

An argument arose between the transportation company and the 
city. Street cars are allowed by the laws of the city to make left turns at 
certain corners. Busses are not allowed to make the left turns. It was 
to the advantage of the car company to have their new vehicles make 
these left turns, but some members of the city council said it would not 
be legal. Assume the statements above to be true. 

Conclusions— 

—A. By the laws of the city, the new vehicle should not be allowed to 
make left turns. 

—B. By the laws of the city, the new vehicle should be allowed to make 
left turns. 

~C. This cannot be settled without further information. 


The list of reasons has been omitted because of lack of space. 
If the suggestions for constructing the test for application of 
principles are followed, teachers will have little difficulty in 
developing a list of reasons for this or similar problems. 

We might construct some exercises which get at the ability 
of students to recognize unstated assumptions and to evaluate 
them by using this same technique but other types of problems. 
For example: 


Mary had been making a study of different forms of government. 
She prepared a report which she gave before her social-studies class. In 
conclusion she said: “There are three types of government in power to- 
day: Communist, Fascist, and Democratic. The government of Russia 
is Communistic. The governments of Italy and Germany are controlled 

Another of these problems was given in its entirety on page 66 of the EpucaTIONAL 


Researcu Butretin for March 16, 1938. 
* See pages 89-94. 
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by Fascists. The governments of England and the United States are 
democratic. I have given facts to show that the governments of Russia, 
Italy, and Germany are really unsatisfactory for the common people. 
If you accept these facts and this conclusion, then it follows that the 
democratic form of government is the only satisfactory form for the 
common people.” 

Now let us assume that the facts Mary has presented concerning the 
three types of government in power today are accurate. Let us also 
assume that these facts actually do show that the governments of Russia, 
Italy, and Germany are really unsatisfactory for the common people. 
Conclusions— 


—A. Mary proved that a democratic government is the only satisfactory 
kind for the common people. 

—B. Mary did not prove that a democratic government is the only satis- 
factory kind for the common people. 

Here we have an illustration of the so-called “indirect argu- 
ment.” Mary has discussed three types of government and 
even if the facts about Communism and Fascism are accepted, 
the conclusion that the democratic form of government is the 
only satisfactory one for the common people does not follow. 


F WE want to appraise a student’s facility in drawing fact 
from assumption, his insight in recognizing which as- 
sumptions are of critical importance in the discussion, his ability 
to organize a proof, and his ability to defend his own beliefs, 
we might do better by attacking the problem indirectly. Let 
us present a problem situation to students in which a conclusion 
has been reached. Let us add a section which will include state- 
ments pertaining to the problem. In this list of statements we 
shall include all of the necessary facts and assumptions which 
lead up to the conclusion, we shall include also some irrelevant 
data and some statements which are inconsistent or contradictory 
to the conclusion. We shall follow this procedure by asking the 
student to read the problem, to study carefully each of the state- 
ments, and to mark all those which are facts F and all those 
which are assumptions A. In Part II, we shall ask the student 
to indicate the assumptions which are most critical if the conclu- 
sion is to be accepted. In Part III, we shall ask the student to 
select from the list of statements all those which prove the con- 
clusion. The following is an example of the procedure: 


Step 1. Describe a problem situation in which a conclusion has been made. 
Step 2. Restate the facts contained in the problem description. 
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Step 3. In almost every situation where a conclusion has been drawn, 
there is some gap between the facts presented and the conclusion. In 
other words, some of the assumptions involved in accepting that conclu- 
sion have not been stated explicitly. These “gaps” or implicit assumptions 
must next be stated. The reader is cautioned to include all those neces- 
sary if the conclusion is to be accepted. 

Step 4. Restate the conclusion. 

Step 5. Introduce some irrelevant statements which unthinking students 
might designate as relevant. 

Step 6. Introduce some statements which are inconsistent with or contra- 
dictory to the conclusion. 

Step 7. Rearrange these statements and present both the list of statements 
and the problem to the students. 


The test here is devised by following these directions: 


John and Paul got on a teeter-totter, and sometime later Paul 
noticed that the board would remain perfectly balanced when their feet 
were not touching the ground. “Look John,” he said, “we must be 
exactly the same weight.” 

List of Statements— 
— 1. John may have been sitting a little farther from the center or a 
little nearer than Paul. 

. If two boys on a teeter-totter weigh the same, then the teeter- 
totter w.!l balance. 

. The board probably was not balanced before the boys got on it. 

. John and Paul got on a teeter-totter. 

. The boys were of about the same height. 

. No matter how the boys sat, it would influence the balancing of 
the board and make it impossible for anyone to say that they were 
exactly the same weight. 

. The teeter-totter balanced perfectly when their feet were not 
touching the ground. 

8. The board was twelve feet long. 

g. The teeter-totter balanced when the boys were not on it. 

—10. Paul said: “Look John, we must be exactly the same weight.” 

—11. Both boys were the same distance from the place at which the 
teeter-totter was balanced. 

—12. John’s clothes weighed just as much as Paul’s clothes. 

—13. The way the boys ‘leaned while sitting on the teeter-totter did not 
influence the balancing of the board. 

—14. John and Paul were exactly the same weight. 

Dives tion No. 1: After reading the problem carefully, study each state- 

ment in the list. Suppose, for the moment, that you personally had been 

present in that situation. Under those circumstances, you would have 

accepted some of the following statements as matters of fact. Some of 
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them, however, you would have regarded as assumptions. By assumption 
we mean a statement that is open to doubt, that may or may not be true, 
and which, in this particular situation, you would not accept as a fact. 
Place the letter 4 in front of these statements. 

Direction No. 2: Which of these statements in the list would you accept 
as facts? Write the letter F in front of those statements. 

Direction No. 3: If you were trying to prove that these boys were of 
the same weight, you would make use of the available facts, but it would 
also be necessary for you to make use of some assumptions. In this par- 
ticular problem, select three assumptions which are very important in the 
logical proof that the boys were of the same weight. 

Direction No. 4: When arguments for or against some proposition are 
presented in newspapers, magazines, speeches, or textbooks, we often 
feel that the discussion could have been made more logical. Statements 
are sometimes put in which are really unnecessary to prove the point; at 
other times, important arguments are omitted; on still other occasions, 
statements are included which actually contradict the conclusion. Sup- 
pose that you were describing this situation and you were trying to prove 
that the boys weighed the same. Which statements would you use in 
your proof? Include only those which you think are necessary, but be 
sure to include all of those needed to prove the conclusion. 


[ Nue first student in Table I,’ Charles Faye, has recognized 


13 correct facts and 17 correct assumptions and is about the 
median of the group in this ability. In this test, he selects all six 
(the maximum) of the critical assumptions upon which the con- 
clusions depended. In organizing his proof, he includes 11 rele- 
vant facts, all six of the critically important assumptions, all 
three of the conclusions, and did not introduce any irrelevant or 
inconsistent arguments. His performance is consistently good 
and reveals significant accomplishment in the objective. 

We notice that Ralph Bestor does unusually well in dis- 
tinguishing facts and assumptions (see Columns 2 and 3) and 
less well in recognizing critically important assumptions. In 
organizing his proof, he has included few facts, has introduced 
only three of the six important assumptions, and has included 
the conclusion in only one of the three problems which were in 
the test, besides bringing in two irrelevant arguments. 

Elizabeth Moore is among the highest in the group in 
distinguishing facts and assumptions. In recognizing those 
assumptions which were most important if the conclusions were 


* More complete results for the entire school in the achievement of this objective are 
recorded in the EpucaTionaL Researcu Butvetin, for March 16, 1938, on page 71. 





